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W^0^5%0:y A tworpart research and evaluation study was 
||f|ic ^ e an el erne n t ary (K-8 ) school district ' s 

pS^implementatibn^^^ instructional management system, the 

fff^Co^ ) , as a means of facilitating 

i||S ; the^ass mastery of 78 district-defined 

|£;§c^ testing system consisted of a data bank of 

objectives, equipment to 
^f^macHiiie? score student answer sheets, and reports detailing student 
£>v^a system was piloted by administrators 

§iMan6 j^e^achers |at f A detailed interview 

• schedule was developed to elicit such information as background in 
r ; *^ the;- district; knowledge of t CMI system, including how, why, and by 

; . uses of the system for testing and instruction; 
l^techni calf problems)? potential uses of the system; and its effects on 
: S ^ s chb i z at ion and'iastruction. Analysis of the responses of 49 

■■X school staff members— principals , resource teachers , teachers , and 
c 1 e r i c a 1 s t a f f from both pilot and nonpilot schools — revealed wide 
^| variation studied. The factors that appeared to 

M|!^beginf iv^ntial|in the nature and degree of CMI implementation efforts 

were identified as: (1) technical 
|£|Tp^ on instructional uses; (3) key 

~^instr^ 

||triin^ incentives. A consistent ordering of 

Kg^ithe ,;f p along these dimen s ions was f ound , and it is 

^C^suggeste^ implementation and use of 

guidance to other 

l^fscho^ implementing their own CMI systems. A 

^vcoding; appended and references are included. (DJR) 



ERLC 



4 ^ 



U.S. OEPARTMENTOF EDUCATION 

Office of Educattonal Research and Improvement 
EDUCATIONAL RESOURCES INFORMATION 

CENTER (ERIC) 
j^Thia document haa been reproduced aa 

received from the peraon or organization 

originating it 

□ Minor changea have been made to improve 
reproduction quality, 

• Pointa of view of opinions stated in this docu- 
ment do not necessarily represent official 
OERI position or policy. 



SCOPE OF INTEREST NOTICE 

The ERIC Facility haa asaigned 
this document for processing 
to: 



In our judgment, this document 
is also of interest to the Clear- 
inghouses noted to the right. 
Indexing should reflect their 
special points of view. 



PATTERNS OF IMPLEMENTING A DISTRICT 
COMPUTERIZED INSTRUCTIONAL MANAGEMENT SYSTEM 



LU 



Janet L. Cr1st-Wh1tzel 
Patricia D. Terry 
Richard Edel stein 
Brian Rowan 



FAR WEST LABORATORY 
FOR EDUCATIONAL RESERCH AND DEVELOPMENT 



Paper Presented at the Annual Meeting of 
The American Educational Research Association 



San Francisco, California 
April, 1986 



-Thl s-work-was-suppor-ted- by-a-con trac t-f rom-the-Nat1 onal -Instl tute-of- 



s£ Education, Department of Education, under Contract No. 400-83-0003. The 

^ contents of this report do not necessarily reflect the views or policies of 

.'• ' the Department of Education or the National Institute of Education. 



BEST COPY AVAILABLE 



Patterns of Iapleeerctlng a District 
Conputerlzed Instructional Manageaent System 



This paper presents results of a field research and evaluation study 
designed to examine the Implementation of a district-based computerized 
instructional management system, the Computer Managed Instruction/3000 
UCMI).^The study was conducted by the Far West Laboratory In cooperation 
wlthj ah '"elementary school dl strict with grades K-8. During the 
1 mpl ementati on process, the researchers undertook a formative eval uatlon of 
.ifie;; use^of the CM I testing system. The system had been selected by the 
district as a means to facility and updating of student 

mastery :pf ^8 dlistrlct-deflned currlcular objectives. The district 
selectedlfoiir schools (two elementary and two middle schools) to pilot the 
CHI- sys ? te'm;;-;;?The objective of this paper Is to describe the patterns of 
use of the CM I system as It was Implemented within these four pilot 
schools* This analysis Is part of a larger study reported elsewhere 
(CrlstrWhltzel, Edel stein 4 Terry, 1985). 

ThlsVproject emerged from a previous study conducted by the 
Instructional Management Program at the Far West Laboratory, a regional 
survey of Instructional management practices In school districts located 
In the laboratory 1 s tr 1 -s ta te region (Northern California, Nevada, and 
Utah). Educators reported computerized Instructional management systems 
as "exempl ary" prac tl ces f or instructional Improvement (Rowan, Edelsteln, 
4 Lealjt 19jB5)JWHowever, the survey also elicited concerns about problems 
of Implementing district-level Instructional management systems. Examples 
of! such con of training of system users, lack of 

assessment, difficulty In adapting systems to the needs of Instructional 
decision makers, and fears of evaluation by teachers {Rowan; 1984, Rowan 
et aUv!985). 



Perspectives 

Concrete guidelines for designing Instructional management systems have 
been provided by the literature on criterion-referenced testing (L1nn, 
Madaus, 4 Padulla, 1982), curriculum alignment (Mllazzo, Buchanan, 4 
Schutz, 1981), mastery learning (Block, 1971; Bloom, 1976), and outcome- 
based education (Spady, 1982). However, less attention has been paid to 
problems and guidelines for Implementing such systems (Fllby, 1984). 

Studies of change and Innovations In schools and districts have shown 
that success of Implementation, Including district-wide administrative 
computing systems, depends upon many factors (Brown, 1984). One Important 
factor -11 s the support of the principal and other key personnel such as 
resource teachers (Brown, 1984). Th& literature shows that organizations 
(including schools) tend to adapt or adopt Innovations consistent with 
their ■ interests and "climate" (Doyle et, al., 1976; Plncus, 1974). 
Therefore, we felt it was Important to examine implementation of this 
district's CMI in the context of each school and its key personnel. 



: CHI and 01 strict Background 

Oak , Knol 1 Elemen tary Sc hool District (a pseudonym), a medium-sized 
district of 18 elementary and 6 middle schools, Instituted a district 
: fcurrl cul urn correl ated with CT8S (California Test of Basic Skills) 
performance objectives In 1983. The district adopted a plan requiring 
teachers ^ students' progress periodically on 78 district 

4 ob jec ti ves ? f or , studen t performance. Teachers had been updating with a 
£"mat*1^^ mastery or non mastery for 

f^he' objectives required for their grade level and subject(s). Some 
^teacheWv for updating, while others used their own 

bestVjudgment on, their students' progress. The CM I system fit Into this 
? larger|di strict Instructional management program by providing a systematic 
; the task of updating. In 
Interviews; we held with staff In the district, there seemed to be a lack of 
distinction between "updatl ng M and the M CMI ." However, si nee the purposes 
and.uses of bottTof these are related and aim toward the same end, we did 
'. not (and really could not by and large) attempt to separate the concepts of 
"CMI" versus "updating" In our data or In our analyses. 

. In 1984 the district purchased "Computer Managed Instruction/3000" 
(CMI), a software package for the HP 3000. The CM I system was designed by 
. a school dl strict In another state. Using Its own objectives, OKESD 
constructed cri ter Ion-referenced tests (CRT's) correlated with CTBS 
objectives. The CMI can generate and machine score CRT's; update student 
records for mastery of district objectives while scoring student tests; 
generate reports on mastery of objectives with Information on Individual 
students, classes or grade levels within schools and across schools In the 
d1 strict; correl a eeobjectl ves wl th Instructional materials and strategies; 
and al low teachers to Input thel r own tests Into a test data bank. 

The CMI pilot Implementation program was Initially planned to start at 
the beginning of the 1984-85 academic year. However, this schedule was 
delayed due to problems In securing the necessary hardware. By the middle 
bf^the school year, hardware (consisting of a computer terminal and a test 
scanner) was Installed In four pilot schools, designated by the district to 
Implement the testing system. While other schools had access to the tests, 
only some chose to use them and the scanning equipment located at the 
district office. During the span of our study, the district utilized the 
CMI feature of district-developed CRT's In mathematics, and we thus focused 
on studying CMI use In mathematics. CRT's for reading and language arts 
were also being developed, but had only begun to be Implemented by the end 
of the school year. . 



Method 

This section briefly describes the data sources used In this study, our 
Interview sample, and method of data analysis. 

Data Sources 

A detailed Interview schedule was developed, which covered such 
Information as background in the district; knowledge of the CMI system 



: ^ regard 1 ng thb w » W hy » and by w horn It was selected; uses of the system for 
~r testing and Instruction; technical problems; potential uses of the system; 
• Maift? ef f efcts fof rtlietsystem on school organization and instruction [see 
% Terry^fcHsl^ a copy of the interview 

3^schedule)iH| Responses to. these interviews served as the data sources for 
% ; tlil s r study; £ A second 1 n tervl e w schedul e was de vl sed for district office 
^personnel 0$&}tt&}:ncn de ta1 1 ed 1 nformatl on concernl ng CM I-rel a ted 
^^decisionmaking, planning, coordination, and communication at the district 
:^ leveT, : but results of these Interviews are not the subject of this paper], 

:^'Sa«p^e';;{\; 

%l ^Through a process of; nomination by district and school site staff, 49 
administrative and teaching personnel from the four pilot schools and four 
€ s1 mi Tar school s not 1 rapl emen tl ng the CM I were sel ec ted for our school -1 evel 
..y^ample. .There .was an attempt to balance the sample with both users and 
: 'non-users^ as well as to incl ude respondents wl th both posl tl ve and 

negative attitudes toward the system. CHI even district-level staff members 
: from : the central office were also Interviewed for a separate analysis of 
district-level Issues]. Interviews lasting approximately one to two hours 
were; held from March through June, 19Q5 with 26 respondents from the four 
•pilot sites (four principals, six resource teachers, 13 teachers, and three 
aides) and 23 respondents from nonpllot school s (four prl ncl pal s, four 
resource^teacners, and 15 teachers). 

•The> sample upon which the present analysl s was based Included the 26 
individuals from the four pilot schools. Table 1 shows the breakdown of 
this sample by school and position. 

.'Z'\!>.''. '•■'}' Table 1 



Interview Sample from Pilot Schools 



Position 


School 
Level 


A 

El em. 


B 

El em. 


C 

Mid. 


D 

Mid. 


Totals 


Principal 




1 


1 


1 


1 


4 


Resource Teacher 




1 


1 


2 


1 


5 


Teacher 




5 


1 


4 


4 


14 


Clerical s/ Aides 






1 


1 


1 


3 


Totals 




7 


4 


8 


7 


26 



The four schools included two elementary schools (A and B) and two middle 
schools (B and C). The principal and the math resource teacher were 
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;1 ncl iided V; 1 the * sampl e ail; each school. ; In addition, a reading resource 
•teacher^ Interviewed. In each of three schools (B, C, D) a 

^tleiric^ entering data Into the 

computer W more teachers In each school 

except School B were Interviewed. In School B, where CM I was not 
Implemented^^^^e a designated pilot school, we had difficulty 

Iflfidlhg; teachers who were willing to be Interviewed. Thus, our sample from 
thatfe school included only one teacher. 

. toto ; Analyst s,' 

^Interv and written summaries were prepared by the 

Interviewers from audiotapes and handwritten notes. Data from the 

>i n ter v 1 e w s were exam 1 ned quantitatively using an Interview coding scheme 
(seelApperi^ of responses In the major 

categories of ^rcel ved CM I purposes, actual uses, potential uses, 

i perceived benefits^ Incentives to use CMI, overall attitudes toward CMI, 
and; ^ Such a quantitative 

description all ows some\;compar1sons to be made across schools and suggests 
some relationships among categories which might Influence the degree and 

: success of > Implementation, attempts. 

v^V In addition, to obtain a more complete description of the pilot 
schools 1 Implementation patterns, we examined the actual Interview 
ftransicripts; and summaries for descriptive Information and Illustrative 
quotations that \ would help^provlde a:m^ of the 

school s>; and Utheir CMI Implementation. Examples of some of the Illustrative 
quotations; from selected teachers at each pilot school are Included In the 
Append ix.^s6me examples are Incl uded in the text to help Illustrate the 
?sense"; of Before- we r present our data, some cautions are In 

order. V While the dlst^ four schools as "pilot" schools, In 

reality only three of themi one elementary school (A) and the two middle 
school s (C and D), actually implemented the CMI system. In addition, as 
the year progressed, a few "nonpllot" schools actually began some degree of 
Implementation as well, which blurred the distinction between pilot and 
nonpllot schools. In the case of these nonpllot schools, the district CMI 
tests were used, but CMI equipment was not located on-site. Teachers at 
those school s had their CM L tests scored at the district office and 
recel ved the same type of CM I reports as did teachers In the pilot schools. 
Even though Implementation attempts thus extended beyond the designated 
pilot schools, we will confine our discussion of Implementation patterns 
here to the original four pilot school s. 



Results 



In presenting our findings on school Implementation patterns, we will 
ii^ of each of the four pilot schools, based on a 

^ - comb 1 natl oni of both quahtl tatl ve and qual 1 tatl ve data analyses. We first 

use of the CM I along 
with comments concerning the general "climate or disposition of school 

and how It fit Into the school 1 s 
In terms of several organizing 
0 themes |t^ detail s of the four schools 1 patterns of CMI 

5lmtf technical procedures and personnel; (2) key 

Instructional and site 1^ In relation to the CMI; and (3) general 

^■attf^ 

s 1 owing school descriptions or brief "portraits 11 are based on 

data summarized In Tables 2 and 3. In this description and in the tables, 
the school s are presented In order of observed degree of CMI Implements- 
tlpns,; First lis School A, an elementary school, followed by Schools C and 
p, theAtwo middle schools, all of which exhibited varying degrees of CMI 
implementation; the fourth Is School B, an elementary school which, as 
noted, di d not 1 mpl ement the CMI, 

EleHientary School A 

^ the most advanced and successful Implementation of 

v; the CMI s^steim,: The stage had already been set at this school for CMI 
Implementation; as School A had previously pilot tested a precursor local 
system of crlterl on-referenced testing wl th goals and objectives similar to 
^ ^ School A's principal had been Involved both In setting 

up that system and 1n selecting the CMI as a district-wide system. 

V School A respondents, Including the principal and others, seemed to 
share a high degree of commitment to the CMI system and Its purposes. 
Elghty-fslx percent of those Interviewed displayed a positive attitude 
towardthe CMI, wltt none indicating a negative attitude and one who was 
uricertiln;;:: As ar group, they percel ved more purposes for the CMI than 
respbnderittjfrom the other schools and they shared a high degree of 
emphasis 'on Its (71%)/ This school was the 

; only a unified focus on a currlciilar purpose; each of the 

other: schools had only an Isolated respondent. even mentioning the 
curriculum Whet) referring to the CM I's purpose. School A was also the only 
school ; with a nearly unanimous agreement^ on objectives as the focus 
of their curriculum content; the other schools 9 respondents indicated more 
^il verse foci, with generally than half using objectives as their 
^ocus/^ that the Instructional philosophy of School A 

personnel was both shared by the staff In general and consistent with the 
nature and purpose of the CMI. This school was also one that explored and 
tried new Ideas, a disposition that undoubtedly facilitated Implementation 
of the CMI as an innovation. As one staff member stated, "Whenever 
anything new is tried In the district, It Is tried here at (A). 11 
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School 



Perceived 
Purposes 



e'ji- ■ - r -'* r ' 



High In m**ber 
;l) Curriculum : 
2) Monitoring, 
Testing, Stan- 
dardization 



Table 2 

Summary Information on Pilot Schools: 
purposes and Use of CMI 



Primary 

Users 



How Much 
Used Over- 
all 



How used 
Mainly 



Degree of 
Actual Use 



Predominant 
Uses 



1) Tchrs. 

2) R. Tchrs. 



A lot 



Use tests, but High 
not self scan 



pZ, i li ' " . .. . 



Low-Mid In Ho. 

1) Testing 

2) Monitoring. 
Standardization 



1) R. Tchrs. 

2) Tchrs. 



Some 



Use tests, & 
self scan 



Moderate 



Instructional: 
Planning*, 
Grouping* .Diag- 
nosis, Pacing 
Testing: 
Mastery*, ft 
Mln. Prof.* 
Assessment 



Instructional: 
Planning, 
Grouping 

Testing: Mastery 
Assessment 



potential 
Uses 



Mid In Number 
1) Diagnosis (431) 
parent Conf . (43X) 
Parent Conf. 



High In Number 

1) Parent Conf.(50X) 

2) District Up- 
dating/Testing (50X) 
Diagnosis (381) 



M1d-Low In Ho. 

1) Testing 

2) Standar- 
dization 



Clerical 



Some 



Use tests, 
with someone 
else scan 



Moderate- 
Low 



Instructional: 

Diagnosis, 

Planning 

Testing: Mastery 
Assessment 



Mid In Number 
1) Evaluation of 
Staff (431) 
District Updating/ 
Testing (431) 



're 



Mid-Low In Ho. 
1) Monitoring 



Clerical 
(Updating; 
not CMI) 



Hone Hot use Low 
tests or scan (Hone) 



f+uigh agreement among school's respondents 



HA 



Low 



Table 3 

Summary Information on Pilot Schools: 
Attitude Categories 



^School n ; 



Overall 
Attitude 



Level of and 

Major Percclve'J 
Benefits 



Level of and 
Major Incentives 
to Use cut 



Oeijree of and 
Major Sources of 
Resistance 



m. c 8 



Positive (86X) 
Hone Negative 



High Level 

1) Instruction 
(D,P,G,P1) (86X) 

2) District Stan- 
dardization 

(71X) 

Focus on Objec- 
tives (71X) 



High Level 

1) Mandated (71X) 

2) Please Principal 

(29X) 

Potential benefits 
(29X) 



Positive (63X) 
Some Negative 
(25X) 
One Uncertain/ 
Unknown (13X) 



Mid Level 

1) Easier to test/ 
Update (63X) 

2) Instruction 
(0,P t G,Pl)(50X) 
Benefit Students 

(SOX) 



Mid Level 

1) Mandated (25X) 
Easier/Faster (25X) 



Positive (57X) 
Some uncertain/- 
Unknown (29X1 
One negative 
(13X) 



Mid Level 

1) Easier to test/ 
Update (71X) 

2) Instruction 
(0,P.G.P1)(57X) 
Expectations for 
teachers (57X) 



Mid Level 

1) Mandated (29X) 
Easier/Fas ter(29X) 



Low Level 

Technical problems/ 
frustrations (571) 
Report/data problems 
(43X) 



4 Uncertain/ 

Unknown (100X) 



Low Level 

1) Instruction 
(0,P,G,P1) (50X) 
District Stan- 
dard1zat1on(50X) 



High Level 
Technical problems/ 
frustrations (BBX) 
Slow processing/ 
reporting (75X) 
Inadequate training 
(50X) 

Takes too much time 
(35X) 

"Other" (38X) 



Mid Level 

Technical problems/ 
frustrations (43X) 
Report/data problems/ 
(43X) 

Too much class time 
(43X) 

Lack of confidence 
In tests (43X1 
"Other" (43X) 



Low Level Mid Level 

1) Mandated (SOX) Too late In year 

(75X) 

Inadequate training 
(75X) 

Too much/too rapid 
change (BOX) 
Technical problems/ 
frustration (SOX) 
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^ ; v CMI^teclinl cal ; procedures and personnel • Although School A did not 
6uti 11 iej;a; cl erl cal / tec hril call support per son to operate the hardware and 
; scm ^ performed such a f unc tl on, 

;thus|ayo1d1h^ evident among teachers at 

^some^c^ the system hardware. Several 

teachers dl 1n usl ng the system (as would be 

expect^ Implementation attempt); however, 

^unl'ife s, School A teachers did not seem to 

^seefsi^ rather as more of a challenge or as 

'fru1&^ on. For example, a representative 

^cbmme^^ It . • • has Its little 

"Idiosyncrasies ^you'd have to work out . • • all that's been pretty much 
Velimlna^ 

^f|;|k^ (RT) who was centrally 

dinyolved|1n; the operation of the CM I played a key role In 

^theKCMI^ In addition to coordinating the technical 

aspects of; ^ focus on 

?1nstru<^ CMI. The RT Interpreted results from the 

CM I report teachers in focusing on 

blnstructlpnal ob jec tl ves In lesson pi annlng; worked col 1 aboratl vely with 
teacher si at each grade 1 evel to coordinate grouping of students for 
1 ri structlon based on their degree of mastery or non-mastery of district 
bb jectl ye;s;f and, in addl tl on, served as a lea ml ng special 1 st providing 
Instructlbn for those groups of students who had not mastered particular 
objectives and thus needed specialized help. The RT operated a learning 
^abfipeclf^cally organized to provide such Instruction. Because the 
"mastery teacher" (RT) coordi nated the CM I, worked wi th teachers to use CMI 
information in^helr* instruction provided extra Instructional help to 
supplement te efforts, School A appeared to correspond 

to thevdescrlption of ''Baker" School District In a study reported by 
Williams and Banks (1984). ; 

The importance of the RT's role In CMI Implementation at School A was 
Illustrated by one teacher's comment that 

(RT) gave us an In service on (CMI) per grade 
level . . . (It was) very effective . . . 
(For ongoing support) that would be (RT) 
again. ... she is very supportive of the 
program and gives extra in-service (Is) there 
if you want extra help . ••all you have to 
do Is go next door and she's willing to help. 

As mentioned earlier, the principal (P) also provided support and 
leadership for the CMI. A teacher commented, "I think It's (the P's 
support) absolutely (that encourages CMI use), and I think that there Is a 
willingness on the staff to cooperate with the administrative point of 
view." Another teacher stated, "(The Principal) Is very supportive of this 
system." This combination of administrative and Instructional leadership 
seemed to be an Important key to the success of the CMI at School A. 

Degree and focus of inplenentatl on—uses of CNI. Teachers at School A 
used the CMI to a greater extent than at any other school ("a lot" compared 
to "some" at the two middle schools and "none" at the other elementary 



school). Their p^ uses were Instructional, especially for 

planning: and grouping— for both of which there was high agreement (both 
86l).V;Coinpared to other schools, there was also a stronger Indication of 
CMI use by School A teachers for other Instructional applications of 
diagnosis and pacing (both 43%). There was also high agreement on using 
the jCMlXfor^ mastery and minimum proficiency (572). In no 

other schools was there such high agreement (and such high agreement for 
•mill t1ple ; uses) among respondents concerning use of the CHI. Apparently, 
their shared support for the CHI and 1 ts mul tl pie purposes were closely 
rel a ted -to ^nd f ac 1 1 1 ta tl ve of their shared multiple uses, and especially 
their : instructional uses, of the CMI. Examples of teachers' reports of 
their CM I Tuses are shown 1n Appendix F. 

Respondent attitudes related to CHI. Not surprisingly, School A staff 
Had; the; most positive att1 tude toward CMI (862 positive and none negative). 
One. teacher stated, "What I really like about it Is . . . that It does 
correlate very; closely with the CTBS . . . I can't think of anything I really 
don't like about It." Also Indicative of teachers' positive regard for the 
system was School A's highest average number of benefits reported per 
respondent (4.3, vs. 3.4, 3.3, and 1 for the other three school s). In 
addition, again their high degree of shared responses 

concerning CMI purposes, Instructional philosophy, and uses of the CHI, 
School A had the highest shared perception of all schools on CHI benefits, 
and especially Instructional Ily related benefits (862 reported benefits of 
diagnosis, planning, grouping, or pacing; 712 suggested a benefit of 
Improving Instruction by focusing on objectives). 

Consistent with this pattern, School A exhibited the least amount of 
resistance and the most perceived incentives toward using the CMI. 
Although respondents seemed strongly Influenced by being mandated to use 
the CMI, School A was the only school to list pleasing the principal (note 
the earlier discussion on the principal's strong support for the CMI and 
his apparent "school climate" leadership) and potential benefits as reasons 
or Incentives for using the CMI system. 



Middle School C 

School C, a middle school, evidenced the second highest degree of 
implementation of the CMI among the four pilot schools. The principal and 
staff, however," did not have the same degree of commitment to CMI, the 
shared Instructional philosophy, the perceptions of CMI purposes, potential 
and benefits, or the CMI uses that were evident at School A. 

The predominant purposes- of CMI reported by School C teachers were 
testing-related:: testing and mastery assessment (75%); monitoring student 
progress (382); and standardization of district measures of mastery (382). 
For example,; Individual teachers mentioned CHI's purpose as "(focusing) you 
on the proficiencies" or as "assess(lng) mastery of skills and objectives 
their curriculum focus: 382 responded that their focus was on objectives, 
while the same number used the text as their curriculum focus. Thus, they 
did not appear to have the same degree of shared instructional focus or the 
shared currlcular purpose of CHI that was evident among School A teachers. 



:S. ;CHI procedures and personnel. When School C's CMI Implementation 
began; ^ coordinating responsibility, which 

; pr 1 ra arl ly vlri v ol v ed sc ttl ng up files for all CMI tests. With no technical 
v support^ 1 abl e, - teachers scanned their own tests. Later, School C made 
•;a : change ^1n \1 ts CMI operation. The reading resource teacher assumed an 
Iris^ which Included advising teachers on using CMI 

information, somewhat si mil ar^^ of the RT In School A« In addition, 

; ) a substitute teacher with two free periods was assigned the technical 
x ; suw tests. Teachers were reportedly relieved when 

r : they^nc^ One teacher seemed satisfied 

the scanning performed by a designated 
$ pers^ Is done by the resource teacher or an assistant, 

? and ^ the te sts. are We 1 1 org aril zed 1 n files and ready to administer." 

Key personnel. As mentioned above, two RTs were involved In varying 
• wiaySilnjSchopl C^ CMI Implementation. The math RT initially coordinated 
the Xte^ the filing system, but did not deal with Instructional ly 

•.;<;;.' rel a ted support to teachers. Appr ox1 ma tel y midway through School C's 1984- 
85 CMI; implementation; the reading RT did take on a more active role In 
providing Instructional support to teachers 1n using CMI Information. As 
already mentioned, a technical support person was added to the staff to 
scan the CMI tests. The Importance of that support was Illustrated by one 
teacher's comment: 

. v' Unless we have an aide to put these tests through 
the scanner, I don't see how a system like this 
could . . . function, and that's just with basically 
two of us (using It all the time) and just a couple 
other teachers Involved once In awhile. I have a 
question 1n regards to how slow will (the) system 
become when more teachers and more schools are 
Involved. 

While the principal (P) gave positive verbal support to the CMI, he did 
not appear to take an active role In Its implementation. One teacher 
stated that "(P) told me I'd like the system" but added that "(P) never 
gets Involved.' 1 

Degree and focus of Inpleaentatl on—uses of CHI. Overall, the response 
of School C personnel was that they used the CMI to "some 11 degree. They 
had the second highest rate of usage: 2.4 different coded uses per 
respondent compared with about twice that much (4.6) for School A staff. 
Both School A and School C evidenced an equivalent total number of 
different uses which was considerably higher than School D, whereas School 
B respondents reported no uses. However, at School C there was less 
unanimity of uses than at School A, with the highest reported uses (at a 
moderate level) for Instructional uses of planning and grouping and for the 
testing-related use of assessment of mastery (each 38%). Thus, there were 

fewer Instructional uses of the CMI at School C than at School A. Appendix 

F Includes examples of teachers' explanations of their uses of CMI results. 

The one area where School C was higher than School A was In perceived 
potential uses of the CMI: School C staff saw mors total potential uses 
(11, vs. 8 for School A), as well as more per respondent (almost 3, vs. 2 
at A). One explanation could be that since School A staff reported more 



actual uses, potential uses by definition would be more limited than at 
other; schdpls^ ^iStaf f members at both Schools A and C saw considerable 
potential ^u^of the CM I for parent conferences '(43% and 502 respectively), 
while f staff. tottery school s did not give such a response. Other 

potential uses mentioned by several School C staff Included testing or 
v updatlhg^;^ diagnosis (38%). One School C 

teacher said Cthat "the CMT be useful for conferences with parents 1n 

providing specific information about student strengths and weaknesses." 
Another teacher thought that 

to 'haye^fe . tnat . . . this school, this 

class ... had problems with these particular types 
of concepts . . .1 think that might be beneficial 
^ to the whole district. 

Respondent attitudes related to CHI. School C also evidenced the 
second highest positive attitude toward the CM I (63$, vs. 86% for School 
A). School C had two staff members (25Z) with a negative attitude, 
while School A had none. While one teacher said, "(I think) the CM I tests 
are good (they) help to assess If a student really knows the 

material , M another teacher who was Initially posits the CM I had 

begun to have some negative feelings: 

; /iWhem I was Initially told about It, I thought . . . 
we're going to have these tests graded, and that's 
great, and tave all this Information back, and It's 
wonderful. I didn't realize that I would have to do 
:. : so ; much. 

The same teacher al so said, M I would like to know more about ... Its 
capabilities ... and I haven't had that question answered." 

^School C staff apparently felt less motivated overall than School A 
respondents to use the CM I (only 25% reported "mandated by district" and 
25% "easier or faster to update" as Incentives to use the CM I). These 
Incentives; were nearly Identical to those for School D (the other middle 
school Kstaff. 

■Surprisingly, even though School C was generally second to School A In 
CMI use and In most areas related to attitude, Its staff had the highest 
apparent res 1 stance of the four schools to the CMI, as measured by our 
coding schem^.^ They Indicated the most ^ of resistance as well as 

the highest average number of: sources per person. Strong sources of 
resistance were! technical problems leading to frustration (88%), slow 
processing o^, re training (50%), the highest 

or nearly the highest response percentages for any school In these 
categorle^^^ es of teachers' comments Illustrating such sources of 
resistance "are given In Appendix F. 

Middle School School D 

:SchooliD, :the second middle school, was the remaining school among the 
pilot schools to exhibit Implementation activity related to the CMI. The 
average number of different purposes per respondent (1.9) was second to 



School A (2.6) but about the same as School D. Reported purposes of the 
■ CMf were diverse, with the predominant purpose being testing-related-- 
te s ti rig ^ arid ? a s se s smeri t of ma s tery (572). One teacher said that the CMI 
"se^s^to be^ based on the CTBS test." Another described the CMI system as 

having them on the computer and having the 
^computer grade^tt^ percent cited district standardization 

of mastery measurement as a purpose. One of these same teachers replied 
that^ tel 1 "What the district expects." Staff was split 

: between objectives and minimum proficiencies as the focus of their 
curriculum One of the teachers said that "they (the district) 

hive; -their ^objectives and (I); fall In line with their objectives," while 
another said;. "He are supposed to be getting them to pass r . . proflciencie 
tb^graduate." ' 

I procedures and personnel . From the beginning of School D*s CMI 
implementation effort, a clerical aide was the key person in setting up the 
system One of the three math teachers used the CMI 

tests ex te of her own scanning. Two other math 

teachers used the CMI to a lesser extent, while a teacher near retirement 
used it very little. One teacher said, "I know that (the aide) uses 
It .i ; s£ most often, but who's actually responsible I don't know." 

^e^ stated, the clerical aide provided the technical 

support ; for CMI opera ti ons. Some of the teachers felt It was Important 
having the ai del as they were rel uctant to scan tests themsel ves. For 
example, one stated: 

The one who does all the inputting and everything is 
(the aide). Now . . . at least one of (the teachers) 
Inputs on and off too. I don't bother. I figure 
I'll probably mess the thing up. 

The math RT had a positive attitude toward the CMI and reportedly was the 
coordinator, as the clerical aide's supervisor, but played a limited role. 
Thus, there was no instructional support to help teachers use CMI 
Information. Teachers were on their own as far as using the CMI in their 
instruction was concerned. The principal's interest in and support of the 
CMI were low. His attitude toward It was not clearly either positive or 
negative. • 

Degree and focus of 1a pie aentati on— uses of the CMI. The overall 
degree of using the CMI was "some." School D staff reported fewer total 
different uses and number of uses per respondent than both School A and 
School C.; The latter figure for School D was half that for School C, which 
In turn was ! half that for School A. Uses were somewhat scattered, with 
those most frequently mentioned being diagnosis, planning, and mastery 
assessment (43S for each). No other uses had more than one response. 
Reported potential uses were likewise lower in number than for Schools C 
and Ai;; wi th the only two men ti oned by three or more respondents being 
evaluation of staff and district testing/ updating (43% each). See Appendix 
F for representative comments concerning teachers' uses of the CMI. 

Respondent attitudes related to the CMI. The general attitude of 
School staff toward the CMI (572 positive) was considerably below that of 
School A (862 positive), somewhat below that of School C (632 positive), 



but considerably above School B (02 positive). One School D teacher said, 
"I 11 kej knowing what the district expects"; "I like the way It's set up . . . 
; there Were some negatives to start with but the way It's going Is very 
good"; and M I like the consistency of It, the problems are consistent from 
te st • to test." Another teacher was negative: 

; I was anxious to try It . . . My reaction was let's 
■V'"::. get this thing underway. Consequently I found 
^ f ! every single bug that was In the program. My own 

proficiency tests are much more Inclusive, . . . rsuch 
more valid, are easier to administer, (and) much 
^ ; ( ; ' m^ to my time. 

The -total number of reported benefits, as well as the number per 
Individual, also were third but close to the number reported at School C. 
Primary benefits Indicated were "easier to test/update" (71%), 
Instructional uses of diagnosis, planning, grouping, pacing (502), and 
benefltsjfor stiidents (502). Standardization of district measures of 
mastery^was^also mentioned as a benefit by some (432). Incentives for 
using; t^e C I were the same as for School C: two responses (292) each for 
"Mandated by d^ and "easier/faster to update." Sources of 

resistance to using CMI were about the same In total number as for School 
C, biit ^somewhat lower (4.4 vs. 5.3) In sources coded per person (both of 
these "figures were higher than for School A). The predominant sources 
Included tec ("that's my main problem 

1 s the ^techn 1 cal a spec t of It"), problems with reports or data ("the matrix 
did not give the 8th graders credit for having passed the 6th grade test" 
and "the; matrix did not agree at all with the printout"), too much class 
';-;t1me^ihvoly^ ("there's an awful lot of work to be done In the 
claslsrooml^ ^ me to take time out of every day to shuffle these"), and 
^ack -pf confidence In tests (see earlier quotes concerning one teacher's 
be^lleifs f 1n the clear superiority of self -developed tests). 

Elementary School B 

Although named as a pilot school for 1984-85 Implementation of the CMI, 
School B did not participate In the pilot effort. The school did have a 
half-time clerk In charge of the equipment and scanning, but the clerk 
reported scanning tests only a couple times for one or two teachers. We 
were ab 1 e to 1 n tef v 1 e w onl y one teacher and a total of four staff at the 
scho^ the clerk. Three of the four reported "monitoring 

student progress" as the purpose of CMI, while there were scattered other 
purposes suggested by single respondents. 

vCHI^ and key personnel. As there was little or no 

use of ?the CMI, there Is little Information other than that the clerk was 
ayai^ tests If teachers would use them. 

Degree and focus of 1»pleaentfit?^n— uses of CMI. The principal did not 
encourage the use of the CMI. One teacher commented that CMI "was not a 
prloHty^bf jthe school. The RT and the single teacher Interviewed both 
reported morale problems at the school which required attention, leaving 



vIIJ^St 5 0 trying out the CMI system. Ir. terms of potential uses of 
the CMI,. few were mentioned (the least of all pilot schools) and no 
potential use was suggested by more than one person. 

c^i"?*?^!?* at * 1tu< ! es related to CMI. There was no respondent from 
School B. with a clearly positive attitude. All four were coded as either 
uncertain or unknown. They did mention two benefits of the CMI- 
Instructional uses of diagnosis, planning, grouping, or pacing and 
standardization of district measures (2 respondents for each), but the 
number of benefits given were much lower than for the other schools. The 
only^lncentlve for using CMIthat was mentioned was "mandated by district" 
12 of the 4). The number of sources of resistance to the CMI was not far 
different from the other schools; but the types of resistance Involved the 

li^!!^ < 3 of 4 >» ™* too much 

cr too rapid change (2 of 4). In addition to reported technical problems (2 
of 4). which seemed a more predominant resistance at the other schools. 
School B staff , because of not using the CMI system, probably did not have 
an opportunity to encounter the technical problems themselves as was the 
case at the other schools. 



■ ' Discussion 

In examining our data related to the Implementation of the CMI system 
at the four designated pilot schools, we found wide variation on all 
factors we studied, from reported purposes to Instructional philosophy, 
actual and potential uses of the CMI, attitudes, sources of resistance, 
perceived benefits, and Incentives to use the CMI. However, we found In 
general a consistent ordering of the four schools along all these 
dimensions. Such patterns can Indicate relationships among the dimensions 
that in turn can suggest how schools might plan their CMI Implementation 
efforts for greater effectiveness. 

•""S iu^ e ?r ee of 1m Plementat1on and In the disposition of the staff to use 
the CMI, Elementary School A was clearly the most advanced and the most 
positive toward the CMI, followed In order by Middle Schools C and D, and 
then Elementary School B, which actually did not Implement the CMI during 
the pilot period we studied. 

We will organize our discussion of these Implementation findings around 
several factors— technical procedures and personnel; focus on Instructional 
uses; key instructional personnel; site climate and leadership; training; 
and attitudes and Incentives— -which appeared to be Influential In the 
nature and degree of CMI Implementation efforts at the Individual school 
• si tes.' 



Technical Procedures and Personnel 

p v Technical and time problems Inhibited smooth operation of the CMI 
system In some schools. In one school to a greater extent (C) and one to a 
lessep de 9fee (D), at least some teachers attempted to scan their own tests 
and typically -found that It required a good deal of time and became 
frustrating. We found that the school s achieving the most complete 



Implementation in the pilot period had designated particular staff members, 
either an aide or clerk or a resource teacher, to operate the hardware and 
peirfo^ tasks associated with using the system (e.g., 

maintaining and distributing files of tests and answer sheets, collecting 
answer shee^ from teachers, scanning these Into the 

computer system, and returning the test sheets or delivering the output 

(C), which began the pilot 
Implementation with teachers performing their own scanning, > later 
designated responsibility for the scanning. Some 

teachers ; relief that they no longer had to deal with the system 

VlwnlWare much time was wasted by 

/teacher^ and^ after school dealing with technical problems with the 

scanner; which caused resentment among teachers. 

;■■ Ais reported In a companion paper on barriers and facilitators to the 

CMI Implementation (Terry, Crl st-Whl tzel , Edel stein, & Rowan, 1985; also 
■v see/Crlst^Whltzel/ et al., 1985), lack of both time and access to hardware 

were^igienerall^ their efforts to Implement CMI. 

When teachers were responsible for scanning their students 1 tests, they 
\barolyj had enough^ deal with the hardware and Its frustrations; let 

; alone 'pi an instruction based on the CMI Information reports they received. 
■Thusi^lacic- of time also Inhibited teachers attempts to use the system to 

enhance their teaching. v 3 



Fbc^ 

'^-^in^ree school s^ teachers did use CMI results for Instructional, 
purposes. Where CMI Implementation was the most advanced, there were more 
refc^ of the Information received on CMI 

• i^poi^ w 1 th the most compl e te Implementation, the resource 

;■ teacher;^ and all teachers reported using the CMI for grouping students and 
pi annlng instruction; several teachers also reported using results for 
diagnosing learning needs and for pacing Instruction. In addition to these 
Instructional uses, School A teachers al so reported testing and assessment 
uses^as ^ell.^ 1^ two middle school s (C and D), about half of the 
teachers reported uses for diagnosing and planning (School D) or for 
planning and grouping (School C). It seemed that when there were both (1) 
more reported Instructional uses and (2) consistency of those uses across 
staf f ,^as at School A/ theire al so tended to be a greater degree of 
1 mpl ementatl on of the CMI. In the schools where staff reported more 
I nstr uctl^ uses, they al so generally percel ved more potential uses of 
: the J system. was School A; It seems likely that Its 

[staff ^members perceived a more limited array of potential uses because they 
reported a ;h1 g her number of actual uses. 

Key Instructional Personnel 

£ !^J^ most effective 

1 nipl eraeiim system, parti cul arl y as It related to 

instruction, had designated a resource teacher to have coordinating 

v resppnslbi 1 1 tle^ In two schools (A and C), this person 

r^iew^ advised them about objectives and 

; skll l^fta' focus on as wel l as possible Instructional strategies. CMI uses 



were greater. In School A where a resource teacher performed this function 
i ■ tne beginning. In School C, a resource teacher began to take a more 

active role In encouraging Instructional uses midway through their pilot 
. Implementation.; In the school (D) where an Instructional resource person 

did not perform such a function and teachers were left to their own 
: Initiatlye to use the information, we tended to sec less instructional 
^application i ;0f CMI Information. 

Si te Cllnate and Leadership 

n ^;Me found that a key condition for a high degree of CMI Implementation 
and its Instructional applications seemed to be a supportive school 
cl .J ma $ fi ' with a strong supportive principal who valued the system and 
-encouraged its by teachers. In such a "supportive school 

*v , ? mate »:«ll or most of the staff share an Instructional philosophy 
consistent with the purpose(s) of the CMI as well as an attitude fostering 
the :expl oration of new Ideas aimed at Improving teaching and learning. 

r For. example^ at the school with the highest degree of CMI 
• Implementation, our data: showed that the principal, resource teacher, and 
nearly all teachers interviewed agreed that a primary purpose of the CMI 
was. to, focus on curriculum content and instructional objectives, and they 
state dt h «t their curriculum was objective-based. A higher degree of CMI 
Implementation thus seemed to be associated with multiple shared purposes 
of the; system by all levels of the school staff and a shared staff focus on 
ob jec tlve -driven Instruction, a condition that seems consistent with the 
nat ure and purpose of the CMI and thus perhaps fad 11 tates CMI use. 

.; In the school s wi th the 1 east evident degrees of CMI Implementation, 
the staff cited diverse and individual purposes for the CMI, as well as 
fewer instructional purposes; the focus of their instruction and their 
overall attitudes toward the system also varied. These schools evidenced 
variable usage of the CMI testing system; no key Instructional leader to 
guide teachers 1n curriculum development and instruction; no organizational 
groups to^assi st teachers with curriculum; more CMI problems cited; and 
more negative or uncertain attitudes about the system. 

Where the principal and/or the resource teacher was fully supportive of 
the system and explicitly promoted its use by all teachers, there tended to 
be a greater degree of acceptance of the system, consensus of purpose, 
coordination of the use of the system by the resource teachers, and 
integration of .the CMI into teachers' classroom management practices. In 
these, schools, a key person -coordinated instructional activities; more 
instructional uses and potential benefits of the CMI were reported; and 
fewer sources of resistance were mentioned. 

Principals^ support of the CMI varied. Where the teachers reported 
that they were not required to use the system by their principals, we found 
nq)usage of the system or less usage than in schools where key personnel 
were supportive of the system. Some principals didn't show much Interest 

;jn the CMI and didn't feel Its use was necessary. In these schools, use of 

;the system was sporadic, or nonexistent. 



Training 

^hlle the CMI technical training provided by the district was generally 
reported to be adequate and the technical support was considered to be a 
key factor In the ability of school personnel to deal with the hardware 
system/ as noted else al., 1985; Terry, et al., 

1985) quite a number of interviewees In the schools felt more training for 
Instructional uses of the CM I was needed. We found that technical support 
personnel, when available at the pilot school sites, and resource teachers 
also played an important role In assisting teachers to use the system 
hardware, a to teachers we Interviewed. However, CM I -related 

training focusing on instructional applications of the system was 
noticeably absent In these schools during the pilot phase. The only such 
1ns true tlonally oriented training In evidence was provided Informally by 
Individual resource teachers at some of the sites, apparently on their own 
Initiative. 



Attitudes and Incentives 

In general, there appeared to be an absence of Incentives or rewards 
for teachers In the pilot schools to Implement the CMI system. On the 
contrary, many teachers seem the CMI Implementation 

: represented a "mandate" from the district, Imposed without any Involvement 
on their part/ findings consistent with those presented for all school 
personneljln the analysis of barriers and facilitators. In addition, CMI- 
related tasks brought added demands on teachers 1 time and thus sometimes 
detracted ^f rom their other Instructional duties and activities. It Is 
1 mpbrtant!; to note that a ma jori ty of the teachers Interviewed had generally 
positive attitudes toward the CMI and Its potential uses; however, many of 
them mentioned negative features of the system. Typically, teachers did 
not perceive positive personal Incentives beyond what ..some of them could 
envision on their own as having a positive Impact on their testing needs 
and their teaching effectiveness. And, while many teachers could Indicate 
at least some Incentive to use the CMI, we found no mention of any evident 
rewards to teachers for exemplary use of the system. 

there seemed to be a relationship among attitudes, perceived Incentives 
and CMI benefits, and the degree of Implementation (especially 
Instructional uses) In the schools. In School A, the staff shared the most 
positive attitude toward the CMI of all the schools; they also saw the most 
benefits in the system ^ focused the most heavily as a staff on the 
Instructional benefits; had the least amount of resistance toward the CMI. 
Al though the other; school s 1 staff, reported the district 
mandate as an ^1 ncentl ve to use the CMI, they were the only staff to be 
motivated f\a^Hw^1 to use the CMI for Its positive potential or to support 
(please) :tl^ principal. • 

The FI ndl ng s in * Perspec tl ve 

Consideration of some general principles regarding attempts at 
i 1 mpl ementl ng 1 nnovatl ons within organizations will help to place our 
findlngs in perspective by pointing out that OKESD Is not unique 1n the 
problems or the outcomes of its implementation efforts, but In fact the 



district operates like many other similar organizations Introducing new 
programs. 

A comparative example: implementation of an Instructional Information 
system In a school district. In a study of a somewhat similar Innovation 
attempt, a school dl strl ct evolved 1 ts Innovatl ve computerl zed 
Instructional mangement system over a period of several years. It might be 
useful to point out some general comparisons of that district's successful 
Implementation with the Initial efforts of OKESD. 

Will lams ; and Bank;(i984) .sidled HBaker" School District which 
Implemented a program somewhat like the CM I system In OKESD. Baker, at the 
t1me^of?:,the study, had been Involved In developing Its CRT system over an 
eight-year period. The system was based on CRTs coordinated with the 
district currlcular scope and sequence, similar to the CM I tests except 
that the 'CRTs were all teacher-developed. Its earlier years, like Oak 
KnolVs pilot phase we studied, Included some resistance from teachers and 
administrators. However, after time the rate of adoption of the CRT system 
by teac hiers was very high. c : 

Operationally, Baker schools showed some similarity to School A In Oak 
Kholl,^ highest degree of CM I Implementation during our 

study. In Baker, there Is a learning specialist In each school who 
provides CRT support for teachers similar to that of School A's resource 
. teacher: 

the learning specialist ... functions as a resource 
to teachers and makes It possible for them to act upon 
the diagnoses Implied by their students 1 responses 
on CRT tests . . . brainstorms with the teachers about 
Instructional alternatives. More Importantly, he or 
she provides extra Instructl onal time, on a pull-out 
basis, for/children who need It. Learning 
specialists coordinate and facilitate the CRT 
testing/ making sure that the teachers review and 
act upon them (Williams & Bank, 1984, p. 276). 

In addition to these support functions, Baker's learning specialists also 
"provide in-service sessions about the functioning of the CRT system for . 
new teachers, and they update experienced teachers about new policies" 
(pif276). This function Is consistent with the suggestion we make that 
resburce teac CMI training. In Baker, the principals 

play a key Instructional support role and are expected to spend 
considerable time In the classroom, as well as to review CRT results with 
teachers. 

^ by Hill lams and Bank (1984) for the success of the CRT 

systemVin Baker district Included: (1) high levels of teacher parti cl- 
patlon (and their earlyjlnyolvement) In formulating the program and of 
f ami 1 1 arl ty w^ the system; (2) program workabl 1 1 ty , with a practical 
stifl^development program providing teachers with skills In acting on test 
resul ts ; ( 3 ) 1 n tegra tl on of al 1 parts of the system, especially support 
fro^ such as the learning and media specialists; (4) 

v Impaction planning which was both regular and collaborative, part of this 



Impact resulting from the fact that teachers were quite articulate about 
tne positive uses of the CRTs. Other key elements of the CRT program were 
Its provision of common goals and expectations; Its encouragement of 
colleglallty In the schools with a culture of cooperation and teamwork; 
and a sense of shared values and commitment to student achievement. 

• • These; characteristics of a successful district Implementation of a 
system similar In many ways to the CMI amplify the model of School A 1n 
OKESD Which showed the most successful CMI Implementation. 

^ A concluding practical perspective. It Is Important to point out that 
effecting an Innovation takes considerable time. Our study encompassed 
only the first hal f-year's effort at a pilot 1 mplementatl on: 1 ts probl ems 
and successes need to be considered In that light. OKESD has had a start 
1n Its CMI implementation and, although Its start was several months later 
than originally planned, such a late start Is probably not unusual given 
the technical nature of the CMI hardware and software and the number of 
potential problems that could (and did) occur as a function of the complex 
technology. In another study of the Implementation of computerized 
management systems In school districts, It was found that It took the 
districts five to eight years to evolve their programs to a relatively 
advanced state (Williams & Bank, 1984). 

In view of the relatively short time span we studied, It Is Important 
to acknowledge the successes of the CMI pilot effort. First of all, It was 
Intended to be a pilot "test": any pilot test Is going to surface problems 
and "bugs". To the extent that the district "surfaced" all the technical 
"bugs" ; \tta^4*Vd1d;;and solved many of them (Including many of those we have 
listed : 1n our report), the pilot test can be considered a success, albeit 
perhaps a qualified one. The district learned a good deal about the 
technical nature of the system and about Its use In selected schools: 
such knowledge will provide a great deal of Information that can be 
used constructively in planning and coordinating any expansions of CMI 
implementation (such as Its extension to reading and language) or In CMI 
Implementation at new school sites. The district did have some examples of 
successful: CMI Implementation and use, particularly at School A, which can 
serve as models to other schools In the district as to how to go about 
Implementing the CMI effectively. 



Significance 

As more districts Incorporate technological Innovations, focus on 
ensurl ngC student mastery of basic skills, standardize their currlcular 
objectives, and align their testing systems with their curriculum In 
efforts to reform their educational programs, there will be increasing 
interest In Implementing computerized Instructional management systems 
such as the one examined 1n this paper. Other districts can benefit from 
; the^exjra^^ 

Implementation and use of such a system can provide guidance to other 
districts just planning or beginning Implementation of their own 
computerized Instructional management systems. 



19 



21 



References 



Block, J. H. (Ed.). Mastery Learning; Theory and practice . 
New York:; Holt, Rlnehart 4 Winston, 1971. 

Bloom, B. Si Human characteristics and school lear ning. New 
r York: McGraw Mil. l9?6. — [ " 

Brown, D. J. '-The change to administrative computing In schools. 
AEDS Journal, Fall 1984, 17-30. 

Cr1 st- W hi tzel , J. L., Edel stein, R, & Terry, P. An Interactive field 
study of a computerized Instructional management system: Final 
" Report. San Francisco: Far West Laboratory for Educational 
. Research: 4 Deyel opment, 1985. 

Doyle, H„ Crlst-Whitzel, J., Donlcht, T. L„ Elxenberger, D. F„ 

Everhart, R. B., McGeever, J. M„ Pierce, D. R., & Toepper, R. M. 
The birth, nurturance and transformation of an educational reform . 
•Tacoma, HA.: Northwest Regional Educational Laboratory 
(Experimental Schools Evaluation Project) , 1976. 

F1 1 by, N. . Technical proposal: Instructional Management Program. San 
Franc Isco: Far West Laboratory for Educational Research and 
Development, 1984. 

L1nn, R. L., Madaus, 6. F., & Padulla, J. J. Minimum competency 

testing:; The state of the art. American Journal of Education , 
1982, 90, 1-34. — 

Mllazzo, P., Buchanan, A., & Schutz, R. E. Methodology for 

analysis of IAI_ district level data bases. Los Alamltos, CA: 
SHRL Educational Research and Development, 1981. 

Plncus, J. Incentives for Innovation In the public schools. Review 
of Educatl onal Research , 1974, 44, 113-144. 

Roberts, J. M. E. Implementation of Innovations In educational 
. organization and instruction. New York: Research for- Better 
"Schools, Inc., 1978. - 

Rowan , B . Instruc tl onal management systems In school districts: 
Promises and pitfalls . San. Francisco: Far West Laboratory 
for Educational Research and Development, 1984. 

Rowan ; B Jv ; Edel ste 1 n , R . , 4 Leal, A. Pathways to excellence: 
. ; What^school districts are doing to Improve instruction . 

, San Francisco: Far West Laboratory for Educatl onal Research and 
Development, 1985. 



Spady, W. G. Outcome-based Instructional management: A sociological 
perspective. The Australian Journal of Education , 1982, 26, 
123-143. ~ ' 

Terry , P. D., Cr1 st-Wh1 tzel , J. L, Edel stein, R., & Rowan, B. 

Implementation of a computerized Instructional management system: 
Barriers and facilitators. Paper presented at the annual meeting 
of the American Educational Research Association, San Francisco, 
1986. 

Williams, R. C. & Bank, A. Assessing Instructional Information 

systems 1n two districts: The search for Impact. Educati onal 
Evaluation and Policy Analysis , 1984, 6i, 267-282. 



APPENDIX 




CODING MATRIX 



revised 6/12/85 



00 Other 



Highest Grade 
~0 Non-teacher 
K - 8 

9 Unknown 



Personal History 

1 -2 Personal Identification number 
10 George 
20. Hubbard 
■ 30 , Ocala 
40 Meyer 
50 Sheppard 
,60 Shields 
70 Slonaker 
80 Pal a 
90 District 

3 - 4 Position/Grade . 

T Teacher 

R Resource Teacher 

P principal 

\ A Aide : 

C Clerical 

E Cwnp Ed 

V Evaluation 

U Curriculum 

5 - 6 Years in district 
99 Unknown 

7 Pilot status of school In which employed 
P Pilot 

N Nonpllot 

8 Level of school In which employed 
E Elementary 

M Middle 

9 Card #1 or 2 
10-11 Blank 

Purposc/Communi cat1 ons ( statement of purpose, use of system) 
Monitoring student progress - updating 
Curriculum - focus on content end Instruction/ objectives 
Testing - CTBS and assessment of nastery 
Evaluation of teachers, teacher performance 
increase Test Scores 

Standardization of district measures of nastery 

•18^i^Bl anicf 



12 ;V 


M 


13 


C 


14 


■0; 


15 


Si- 


16 


/S 


17. 


D 



CODING MATRIX p. 2 

Curriculum/Content 

20 Content Focus: Choose the one that applies. 

0 On objectives (objective-driven Instruction) 
T On text or other 

X Unknown, not mentioned 
M On minimum proficiencies 

21 Change in textbook use 

Y Yes, there has been a change 
N No, there hasn't been a change 
X Not mentioned, unknown 

22 Change In scope of curriculum 
M More extensive 

1 Less breadth 
N No change 

U Coder uncertain, protocol unclear, respondent undecided 

a Not mentioned, unknown 

23-25 Blank 

Testing J 

;- : 26 Change In amount of testing with iMI/updatlng 
M More testing 3 
L Less testing 

N No change 1n amount of testing 
U Uncertain, unclear, undecided 
X Not mentioned, unknown 

27 Difficulty of testing with CMI/updatlng 
H More difficult to test 

L Less difficult, to test 

N No change In testing 

U Uncertain, unclear, undecided 

X Not mentioned, unknown 

C Conditional 

28 Type of tests used 
■ C CMI 

U Unit tests from texts 

0 Teacher's own tests 

X Unknown 

F Unit tests with teacher's own tests 

G CMI with unit tests 

H CMI with teacher's own tests 

0 CMI with unit and own tests 



29 



y het J er student performance on the CTBS objectives affects grading 



^Srl'l R v ; ' • CODING MATRIX ' 

v -lhitrucitort applications - test results and/or matrix: Non-CMI 

•^:^^6*:r. ; 37.; • How/test Information Is used: code each of the 
■iff; r : -:^ ^following ..that -apply; put In X when It doesn't. 

|^ 30 H G ' Grouping/Placement 

/ v ; ;' 5 ';.y;v-.v32v6-^ ; - Diagnostics for individuals ' . 

tfifP.tt^., P ricing : 

^>.Wv; '.;i.U^'35 C A,?.? Hin liauim -..cdnpe.tency r 'f or .' pr emoxl on/graduail on 
Other appl 1 cat 1 ons of test resul ts and/or matrix: Non-CHI 

».!•> : ? •• .••..-'.'»••■ , ... ;. . . • 

; 37 • S Student conferences . 

■;\ Y//^38'* i F : Family tutoring 

39 E - Evaluation . 

p. ' "40 : ' ; -v . ' : Blank 
Instruction ^ 

p-^-Al - ;51 How CMI Information 1s used: code each of the 

^ ^ 1n X when 1t doesn't. 

i ^: 43 D ^ • Dlagnostl cs for Individuals 

4 j£ £ • ''-J M k Mastery :• assessment 
|^|^|r^:;^C^:M^tia^ proficiency/competency 

48"' *- ; Sk ; > ' Student conferences 
* 49 F Fam1 lyr tutor 1 ng .' 

50 E .Evaluation 

^Ms51?t; 52*$Blink\£-~ -k .. : t 'v..;. ; ' On 



CODING MATRIX 



P. 4 



f Updating 




Updating other areas 

C ^ CMI 



MM Matrix , 
N Nothing 



Z> Subject area not applicable 
X.:? Unknown 



54 



Present system used by person for updating oath only 



M; Matrix 
N Nothing 

Z Subject area not applicable 
X Unknown 



': '-' Tlme'-Use ;^ ! ••: 

^^^^ ^ 5 on CMI, updating, etc. (1n hours per month: 

'*v"vi.-.-"' ■ multiply dally &y 20. weekly by 4) 

'■}l : f'7>>-$/:t--';: . 1 r 0 - 5 hours/month • 
2 6- 10 : 
■ : 3-r^ll-.- ;15' " ' 
4 16* 

i'J^y-.. 9 •;. .Unknown' \ 

V58 Change 1n time use 

••V' V - ^. ; 'M More • . 

•■■ .; : L less . ,. . 

N No change. ■, 

U Uncertain (Interviewee) 

X Unknown (to coder) 

■''■Y'V--^ \ C Conditional ' 

^ : - v:S^:-: ; 64; : :y Tasks.. > :: 

59 Preplanning . 

:re^/6/":(6or.;;;- : "S : .V ; Scanning 

!ft'& r fr,'61 R Recordkeeping and clerical 

$5^-: l ; -J$4-H ':--s ^.+.y?Jcf. ' ". 
i$^&5?62; ^Scoring v 



^-^55 56 Bl ank 



e^ev63'.' : * A -< ; M^\ Meetings 
, • ' 64 0 Other 
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CODING MATRIX 

p. 5 

Card 2 

Potential Usage 

12 - 24 Possible uses for the system: code each of the 
following that apply; put In X when It doesn't. 



12 


U 


Updating/Placement 


13 


6 


Grouping 


14 


• i- 


Lesson Planning 


15 


:•. P 


Diagnostics for Individuals 


16 


P 


Pacing . 


17 


M 


Mastery Assessment 


18 


C 


Minimum proficiency/competency 


19 


' 2 


Parent conferences 


20 


s 


Student conferences 


21 




Family tutoring 


22 


E 


Teacher evaluation 


23 




Testing 


24 


0 


Other 


25 - 


26 


Blank 



Technical 

-27 Primary users of hardware 
T; Teacher 
P Principal 



R 


/ Resource teachers 


C 


Clericals 


0 


Other 


A 


Aide 


X 


Unknown 


NW 


Nonpllot 



CODING MATRIX 

p. 6 

t echnical (continued) 

28 Location of equipment 
T Teacher workroom 





L 


Library 




0 


Office 




P 


Principal's office 




2 


Other 




X 


Unknown 




N 


Nonpllot 


29 - 


34 


Technical Problems: code each that applies; put X when 






1t doesn't. 


29 


H 


Hardware 


30 


S 


Software 


31 


o 


Technical Output (e.g Inaccuracies, missing Info) 


32 


c 


System compatibility (Incorporating other systems Into CMI) 


33 


F 


Test format (Including legibility) 


34 


Q 


Test quality (too hard, too easy, too short, proper alignment 


35 


U 


Problems: Source unknown 


36 - 


37 


Blank 



Report Use : Is the output OK? Use the following codes: 

/ 38 - 4i H Helpful 

S Satisfactory 

N Some problems 

M Many problems 

X Unknown, no comment 

38 For teachers: format 

39 For teachers: content - Information 
. 40 ■■' For. parents ■ 

; : : 41 • For students • 

•gV^;;420 43" Blank - 0 



CODING MATRIX 

: f< ,-. p.7 

Training 

44 Wes tralnlng/lnservlce vscelved? 
Y Yes 

N NO 

X Unknown 

45 Was the training effective? 
H Helpful 

S Satisfactory 
f N Negative aspects 
X Unknown . 

46 Is there ongoing support? 

H ingoing support: Helpful and of high quality 
S . Ongoing support: Satisfactory 
N ^ Not received or negative aspects 
X ,; Unknown '•. 

47 - 48 61 ank : 
Admlnl strati ve I ssues 

49 Has respondent personally Involved In the selection and 
development process of the CMI? 

■'^•Y^YesV; •'■ 

.y . u No 

U Unknown, uncertain 

50 What Is respondent's attitude towards the CHI/updating system? 
p Positive 

N Negative 

U Uncertain, unclear, undecided 
X Not oentloned, unknown 

51 - 52 Blank ' 
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ERIC 
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CODING MATRIX 

P.B 



Sources of Resistance 



53 - 67 ; Put P as a personal concern of the respondent, 
Y as a concern of others, 
N In categories not reported. 

53 Too much and/or too rapid change 

54 Doubt about long term commitment to program 

55 Too late In year for Implementation 

56 Technical problems leading to frustration/avoidance 

Lack of support from: 

57 respected teachers 

58 teachers association 

59 principal 

60 district 

61 Processing/reporting too slow 

62 Problems with report or data (wrong names, misplaced Info) 

63 Fear of evaluation 

64 Inadequate training 

65 Too much paperwork 

66 Too much class time 

67 A lot/ too much time 

68- Coercion mandated by district 
69 £; Lack of conf 1 dence In tests 
70 ^ ;f Teachers not asked for Input 

7 1 xi Coordl na tl on and supply of materials 

72 7 ^OXXier 



CODING MATRIX 



School Organisation : (Card 3) 



p.9 



12 



13 

13 
14 
15 

16 
17 
18 

19 
20 

21 

22 

23 



24 



25 , 



Collaboration with colleagues 

M. More :/ ■ 

I Less: 

N No Change 

U Uncertain 

X Unknown 

21 for these numbers use the appropriate letter Including: 

N No change 

X Unknown 

Changes In eeetlngs 
A : Yes , change 1 n amount 
T Yes; change In type 
\ : F Yes, change In frequency 

Change In school structure 
S New structure, e.g. lab 

G Grade level collaboration 

I Inter-grade level collaboration 

0 More departmentalization 

C More self-contained classes 

0 Other 

Class schedule changes within the school 

Y Yes, there have been changes 

M No, there haven't been any changes 

X Unknown 

Staffing changes 

R Yes, changes in staffing responsibilities 
A Yes, adding staff 
0 Yes, decreasing staff 

X Unknown 

Changes' In teacher relations with students 
A V Yes, positive 
B- Yes,- negative 
"No change o 
r X (^Unknown v; 

: Changes^ In teacher relations with parents 
A Yes^ positive 
B1 ; Yes; negative 

&N : -; ;;; *Norchange : . :: ':-- 
j X ^ Unknown 



ERIC 
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CODING MATRIX 
(Card 3, continued) 

Blank 



Possible benefits to using the system: code each of 
the following that apply; put 1n X when 1t doesn't. 

Easier and/or faster to test 

Easier to update 

Standardization of district measures 
Administrative needs 

Instruction (planning, diagnostics, pacing, grouping) 
Students 
Parents 

(makes teachers aware of) specific requirements and district 
expectations y 

Accuracy of assessment/objectives measures 
Improving Instruction by focusing on objectives or proficiencies 
Blank 

Motivation/Incentives 

40 - 44 Why do teachers use updat1ng/CMI? 
40 P To please/appease the principal 

: ;; : : '-: Ai ■i- M Mandated 

y 42 C Curious/ Interested 
; - 43 - B Potential benefits 
y?^4^\^.J Easier and/or faster 



26 -27 

^Benefits 
- 28 - 37 

28 T 

29 U 

' 30 D 

'^■^■':h 31 a •'• 

' / 32 ;.' I \ 

• 33 • S 

•. 34 ' , P ■ 

35 R 

36 y 

• ; ; ;: ;--37 > F 

38 - 39 



CODING MATRIX 
(Card 3, continued) p - 11 

45 - 46 Blank 



Procedural Aspects 

47 How much has the person used CHI? 
L A little 

S • -•.,Some A " 
MA lot, much 
N Not at all 
X Unknown 

48 How has. the person used CMI? 

L Uses tests without scanning 

S Uses tests with someone else scanning 

M Uses test and scans himself 

N Not at all 

X Unknown 

50 - 51 Blank 

Recommendations for future use 

52 i C New, clearer test copies 

53 K One key person responsible for scanning on site 

54 T More technical training 

i 55 I Training for Instructional uses 

56 A More administrative support In building and district 

57 0 Other 
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